feat(records): stream-type-aware concurrency limits by andersfylling · Pull Request #2688 · cognitedata/cognite-sdk-python

andersfylling · 2026-06-15T12:28:39Z

Why

The Records API enforces hierarchical rate limits that differ between mutable and immutable streams. The SDK needs to model these so that concurrent requests stay within budget without relying solely on server-side 429 retries.

Key constraints from the API:

Write budget is shared across ingest/upsert/delete, same for both stream types.
Query budget is shared across all read endpoints, but mutable streams get 3x the concurrency of immutable.
Retrieve and aggregate each have a dedicated budget checked in addition to the shared query budget — a single request must pass both.

Operation	Semaphores acquired (in order)
list / sync	`query_{type}`
retrieve	`retrieve_{type}` → `query_{type}`
aggregate	`aggregate_{type}` → `query_{type}`
ingest / upsert / delete	`write`

What

RecordsGlobalConcurrencyConfig with separate semaphores per budget tier and stream type (7 total: write, query/retrieve/aggregate × mutable/immutable).
HierarchicalBoundedSemaphore that acquires multiple semaphores in order for retrieve/aggregate, with cancellation-safe rollback and exception-safe release.
RecordsAPI._get_semaphore(operation, stream_type) routes to the correct semaphore(s).
Validation: dedicated budget must be ≤ shared query budget, enforced at construction and on every setter.

🤖 Generated with Claude Code

Model the Records API's hierarchical rate-limit budgets for mutable and immutable streams. Write semaphore (20) is shared across both stream types. Query, retrieve, and aggregate each have separate mutable and immutable semaphores matching the API's documented limits. Retrieve and aggregate have dedicated budgets checked before the shared query budget (both must pass). The semaphore helpers on RecordsAPI are wired up for all three tiers, ready for use by endpoint implementations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Single method handles all operation types (write, delete, query, retrieve, aggregate) with stream_type routing for read operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…rcement Retrieve and aggregate must pass both their dedicated budget AND the shared query budget. HierarchicalBoundedSemaphore acquires multiple semaphores in order (dedicated first, query second) and releases in reverse. _get_semaphore returns this composite for retrieve/aggregate operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

codecov · 2026-06-15T12:57:57Z

Codecov Report

❌ Patch coverage is 99.10394% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.74%. Comparing base (7934665) to head (4e3059f).
⚠️ Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
tests/tests_unit/test_utils/test_concurrency.py	98.36%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2688      +/-   ##
==========================================
+ Coverage   93.65%   93.74%   +0.08%     
==========================================
  Files         498      498              
  Lines       50391    50941     +550     
==========================================
+ Hits        47196    47754     +558     
+ Misses       3195     3187       -8

Files with missing lines	Coverage Δ
cognite/client/_api/data_modeling/records.py	`100.00% <100.00%> (ø)`
cognite/client/_sync_api/data_modeling/records.py	`100.00% <100.00%> (ø)`
cognite/client/utils/_concurrency.py	`94.19% <100.00%> (+1.39%)`	⬆️
...ts_unit/test_utils/test_concurrency_api_routing.py	`100.00% <100.00%> (ø)`
tests/tests_unit/test_utils/test_concurrency.py	`98.89% <98.36%> (-1.11%)`	⬇️

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…maphore routing - HierarchicalBoundedSemaphore: acquires both, releases both, releases on exception, limits concurrency to the min of both semaphores - _get_semaphore: returns plain semaphore for write/delete/query, HierarchicalBoundedSemaphore for retrieve/aggregate, wraps the correct dedicated+query pair, and shares the query semaphore Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…eption-safe Two bugs found by adversarial testing: 1. Cancellation during __aenter__ (e.g. via asyncio.wait_for timeout) leaked already-acquired semaphores — now rolls back on any BaseException during acquisition. 2. Exception in one semaphore's __aexit__ skipped releasing the rest — now continues releasing all semaphores and re-raises the first error. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Raises ValueError at config time (constructor and setters) if a dedicated budget (retrieve/aggregate) exceeds its corresponding shared query budget, since the hierarchical semaphore would never use the extra slots. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Each setter validates only its own dedicated-vs-shared relationship instead of re-checking all four pairs. Shared setters (query_*) validate against existing dedicated values; dedicated setters validate against the current shared value. Constructor uses the property setters directly, setting shared budgets first so dedicated setters can validate. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Each setter passes its own name=value as a kwarg override. The method resolves each budget from overrides or self, then checks all four dedicated-vs-shared pairs. Init assigns directly and calls it once. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Setters are now just _validate_budgets(name=value) + assign. The validation method handles both the frozen check (from the override keys) and the budget invariant in one call. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- ruff format: wrap long ternary expressions - mypy: add type: ignore[override] on _get_semaphore (wider signature than base class, intentional) and type: ignore[arg-type] on test-only BoomSemaphore fakes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…client Three tests that push a HierarchicalBoundedSemaphore through the actual SDK _post → _http_client._with_retry → async with chain: - successful request acquires and releases both semaphores - HTTP 500 error still releases both semaphores - concurrent requests are limited by the tighter semaphore Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Seven tests exercising the exact _get_semaphore → _post chain that list/retrieve/aggregate endpoints will use, for both mutable and immutable streams. Verifies correct semaphore types, values, release after request, and that retrieve shares the query semaphore with list. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

andersfylling force-pushed the feat/records-concurrency branch from 9360953 to 889fed8 Compare June 15, 2026 12:34

andersfylling and others added 4 commits June 15, 2026 14:37

refactor(records): make write/delete semaphore mapping explicit

d6cc7e6

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

refactor(records): unify semaphore routing into single _get_semaphore

10edd2b

Single method handles all operation types (write, delete, query, retrieve, aggregate) with stream_type routing for read operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

refactor(records): use enum values in _check_frozen calls

47a08fe

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

andersfylling and others added 11 commits June 15, 2026 15:06

refactor(records): hoist write_op and query_op above match statement

c51c4e8

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

fix: docstring indentation for custom-checks parser

170a2dd

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

andersfylling mentioned this pull request Jun 16, 2026

feat(records): add list (filter) endpoint with read data classes #2680

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(records): stream-type-aware concurrency limits#2688

feat(records): stream-type-aware concurrency limits#2688
andersfylling wants to merge 16 commits into
masterfrom
feat/records-concurrency

andersfylling commented Jun 15, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 15, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

andersfylling commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Uh oh!

codecov Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

andersfylling commented Jun 15, 2026 •

edited

Loading

codecov Bot commented Jun 15, 2026 •

edited

Loading